Picture-based video indexing system

ABSTRACT

A video indexing system uses pictures representative of a recorded video program to assist a user in determining the contents of a recorded medium without having to view the program itself. The pictures preferably represent segments of the program which are spaced apart in time, enabling a relatively small number of such pictures to characterize a lengthy program, and are presented in separate windows on the screen of the same device used to display the video program. The pictures may include still or moving imagery. A viewer optionally may select a particular picture with a pointing device, to commence replay of the recorded program from that period in the program, or to recall stored audio information so as to assist in identifying the selection. The picture information may be stored on the same medium as that used to record the video program, or a different medium may be used.

CROSS-REFERENCE TO RELATED APPLICATIONS

This is a continuation of U.S. patent application Ser. No. 10/417,300,filed Apr. 16, 2003, which is a continuation of U.S. patent applicationSer. No. 09/191,573, filed Nov. 13, 1998, now U.S. Pat. No. 6,574,416,which is a continuation-in-part of U.S. patent application Ser. No.08/556,746, filed Nov. 2, 1995, now U.S. Pat. No. 6,526,219. The entirecontent of each application and patent is incorporated herein byreference.

FIELD OF THE INVENTION

The present invention relates generally to video recording, and, moreparticularly, to a system for storing and displaying “thumbnail” imagesrepresentative of the contents of a video recording medium, therebyenabling a user to locate a particular section rapidly and conveniently.

BACKGROUND OF THE INVENTION

As the popularity of video cassette recorders (VCRs) has spread, manyusers have amassed extensive libraries of recordings, representing awide range of program materials collected over an extended period oftime. In most cases, users are faced with the alternative of eithermaintaining detailed records as to the contents of individual tapes, orsorting through an entire collection until a desired tape is found.Since the rigorous cataloging of tapes has little appeal to users, mostusers simply resort to notes affixed to individual cassettes, or theywrite descriptive information on a tape label. However, once the numberof tapes in a library exceeds ten or fifteen units, this method formarking becomes unmanageable. Moreover, in the future, new, morecompact, video recording media, such as smaller cassettes, magneticdisks, and recordable optical discs or “videodiscs,” are likely to makesuch marking methods increasingly impractical.

SUMMARY OF THE INVENTION

In contrast to existing video indexing arrangements which aretext-based, this invention uses pictures representative of a recordedvideo program to assist a user in determining the contents of a recordedmedium. The use of pictures enables the index to be generatedautomatically from the recorded program material itself, therebyminimizing or eliminating the need to enter textual or other identifyinginformation. Not only is suitable picture information already availablein the program material being recorded, but it also is clear that even asmall number of pictures may convey significant detail about thecontents of a particular program.

Broadly, given a video program to be recorded, a method of indexing thatprogram in accordance with the invention, includes the steps ofseparately storing picture-based information representative of theprogram and later displaying the picture information without having toview the program itself. An index according to the invention may begenerated after as well as during the recording process. Considering avideo program as a sequential set of images, the invention prescribesseparately storing information representative of a small subset of theimages and displaying images from this subset as a method of identifyingthe contents of the video program. Preferably, the pictures representperiods of the program which are separated in time, enabling arelatively few such pictures to characterize an otherwise lengthyprogram. By way of example, 12 pictures, each representing segments inthe program which are spaced apart in time by 10 minutes, may be used toidentify and/or index a 2-hour movie. The same display device used forviewing the program material may be used to display the pictures, whichpreferably are presented in separate windows on the screen. Inalternative embodiments, the pictures may include still or movingimagery and, as a further option, the viewer may select a particularpicture to commence replay of the recorded program material associatedwith the picture selected, or initiate playback of stored audioinformation associated with the picture selected as part of identifyingthe chosen program segment.

The picture information may be stored on the same medium as that used torecord the video program, or a different medium may alternatively beused. To reduce storage requirements, the information is preferably datacompressed then de-compressed for display purposes. To derive theinformation used to generate the pictures, the video program may besampled on a periodic basis or distributed and received with integratedencoded picture information, which then may be stored separately.

To ensure that the pictures themselves adequately represent the recordedprogram, aspects of the program may be analyzed prior to the extractionof the picture information to ensure that blank segments, commercials,or other unwanted information is not stored inadvertently, instead ofpictures representative of program subject matter. As a further aid tothe characterization of a recorded program, the system may automaticallystore pictures associated with the beginning and/or ending of aparticular program or recording session.

In terms of apparatus, the invention may either incorporate a videorecorder and a display device, or alternatively, use existing equipment,such as a VCR or disc drive, for recording purposes. A televisionmonitor, computer screen or any other suitable display device may beused for viewing. In conjunction with such hardware, the inventivesystem includes an input for receiving a video program to be recorded bythe video recorder, means for deriving and storing picturesrepresentative of time-separated segments of the program, and means forformatting the stored pictures for viewing on the display device. In acomprehensive embodiment, the hardware includes a picture memory (suchas a battery-backed-up random-access memory) for storing the imagesassociated with each of the time-separated segments of the program, aframe buffer in communication with the picture memory and displaydevice, and video display generation means for outputting the contentsof the frame buffer to the display device in the form of a video signal.To convey motion in different windows on the display device, the picturememory stores not one but a series of images associated with each of thetime-separated segments of the program, and further includes means forperiodically updating the frame buffer with individual images from aparticular group, so as to convey motion. In this embodiment, whichsupports moving pictures in multiple windows, a multi-channel tuner maybe added and sequenced so that pictures representative of the videoprograms available on each of the tuned channels will appear in theseparate windows, rather than indexing information.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a drawing of a picture used to discuss its active area andstorage requirements, in accordance with this invention;

FIG. 2 is a drawing which shows the picture of FIG. 1 in one of aplurality of windows generated by the hardware of the invention; and

FIG. 3 is a block diagram of the system according to an apparatus aspectof the invention.

DETAILED DESCRIPTION OF THE INVENTION

Broadly, this invention automatically generates a display of picturesrepresentative of program material stored on a video recording medium.Although the term “video” is used throughout this specification, itshould be recognized as including all forms of electronically recordedimagery, whether or not compliant with a standardized format, includingexisting video formats such as NTSC, PAL, SMPTE Standard 240M HDTV, andso on. Operationally, then, given a recording medium capable of storingvideo information, whether in the form of a tape or disc, whethermagnetic, optical, etc., the user accesses the medium in an appropriateplayback system, and, on an associated display device, there appears oneor more “thumbnail” or “vignette” images representative of the materialrecorded on that medium. In the preferred embodiment, a plurality ofwindows appears on the associated display device, each window includingimagery associated with time-separated portions of the recorded programmaterial. For example, the invention may be configured so as to generatean array of windows on the screen (for example, 9, 16, or some othernumber), each window representing a time-specific portion of the programmaterial, but the contents of each window preferably being separated intime by several minutes, thereby enabling a relatively small number ofsuch windows to represent a more comprehensive amount of recordedprogram material, even hours worth of such programming. To be morespecific, a system according to the invention could generate, forexample, 20 windows, each representative of portions of the recordedprogram material time-separated by, say, 15 minutes, thus enabling the20 windows to be indicative of 5 hours of recorded program material. Aswill be described in much greater detail below, the gap in time betweeneach window need not be fixed, but instead may be chosen in accordancewith some other parameter, such as the beginning or ending of arecording sequence. As will also be described below, the imagery in eachwindow need not be a still picture, but may include motion, preferably afew seconds of images related to a localized time slice. Although theinvention should find immediate applicability in the determination ofthe contents of an unmarked or inadequately labeled video cassette, (inwhich case the invention would preferably be incorporated into a videocassette recorder (VCR)), the invention is not limited to this endapplication, and may be used in conjunction with the identification ofelectronic imagery in disc form, for example, for use with anappropriate reader interfaced to a personal computer or work station,including multimedia CD-ROMs, and so forth.

Generally speaking, then, when video program material is being recordedin a system equipped with the present invention, information used togenerate image windows of the type discussed above is also recorded,preferably on a periodic basis and in digitally compressed form, asfurther described below. Although the information used to generate theimage windows is derived from the program source material, differentalternatives are available, according to the invention, for derivingsuch material and for the storage thereof. For example, an incomingsource signal containing program material to be recorded may be sampledon a periodic basis, and these sample images may be digitally compressedand stored for use during playback of the material. However, theinformation used to generate the window imagery either may be deriveddirectly from the video signal or data stream associated with theprogram material itself or, alternatively, broadcasters may cooperate bytransmitting program material including supplemental information whichmay be used to generate image windows associated therewith, either in anunused portion of the signal for the program material, such as during ablanking interval, on a separate, unused channel, or by otherappropriate means. In terms of storage, the information used to generatethe window imagery either may be stored separately from the recordingmedium used to store the program material itself, or, alternatively, maybe stored along with the program material on the same medium, either inpredetermined areas or interspersed with the program material. Thetradeoffs associated with these various options will become more evidentin light of the descriptions which follow. As an alternative topicture-based indexing in progress with the recording of video programmaterial, a fully or partially recorded medium may also be indexedaccording to the invention on an off-line basis.

Before considering the various alternative implementations, storagerequirements first will be discussed in conjunction with applicablecompression techniques. In FIG. 1 there is shown a picture having anactive area 102 comprising an image area having dimensions of 640×480pixels, which represents a typical NTSC 2-field frame based on squarepixels. One of skill should readily appreciate the applicability of theinvention to video standards having different configurations or imagedimensions in pixels. In order to reduce storage requirements inconjunction with the invention, since the subject matter contained inthe central area of the image most likely will carry the greatestsignificance, a band around the outer perimeter of the image need not besampled, resulting in a sampled area 104 of 576×432 pixels. This assumesthat 24 pixels on the top and on the bottom of the image and 32 pixelson either side are disregarded, although bands of different widths maybe used according to the circumstances. In one alternative, the systemmay include hardware for “recognizing” appropriate subject matter, inwhich case the sampled area may form part of a moving window which movesaccording to actions of the subject matter.

Given a sampled area of 576×432 pixels, if every sixth line is thenconsidered for storage, the result is 72 lines per image, and, if 96equally spaced apart columns are sampled per line, this results in 6912total pixels. If three bytes are used per sample, two bytes associatedwith chrominance information and one byte for encoding luminanceinformation, this results in 20,736 bytes of information, withoutresorting to any compression techniques. However, utilizing availablecompression engines such as JPEG, a compression ratio of 16:1 is readilyachievable, resulting in 1,296, or approximately 1,300 bytes for asampled frame. Preferably, to this will be added a number of bytes whichmay be used for the purposes of identification, including analpha-numeric label, location tag, time/date stamp, and so forth. Assuch, an estimate of 1,500 bytes per picture may be more appropriate inpractice.

FIG. 2 illustrates one possibility for a display generated by thehardware to be described, according to this invention. In this example,the picture or frame of video information just discussed with referenceto FIG. 1, having previously been stored, is now recalled and displayedin one of many windows so as to facilitate the indexing of the videoprogram. Given a display with a “safe action area” of 432 lines eachhaving 576 pixels (which is, again, a sampling area associated with atypical NTSC frame having square pixels), if each thumbnail image to beshown is allocated a display area of 144×108 pixels, sixteen such imagesreadily may be displayed, each having a framing border and optionallyproviding space for one or more lines of textual information. Evidently,depending upon the video standard, amount of available resolution, andthe number of such vignettes to be displayed, the number of windows maybe varied from at least one to more than the number exemplified by FIG.2, again, depending upon the circumstances.

As mentioned, this picture information either may be stored apart from,or in conjunction with, the same medium used to store the programmaterial as it is being recorded. Assuming that such information is tobe recorded on a video cassette along with the program material, variousalternatives are available. Particularly, in light of suitable datacompression, the picture information may be stored on an on-going,progressive basis, along with the program material, for example, in thevertical retrace interval, although other alternatives are possible,such as the use of other synchronization periods within the videosignal, or the control track, particularly for the recording of indexingpointers as discussed below. However, by using the vertical interval, anumber of non-visible scan lines may be used to record serial datarepresentative of the pictures, in much the same way thatclosed-captioning information is recorded. Disregarding horizontalblanking, a scan line represents an available time period ofapproximately 50 microseconds and, if this serial information is writtenat a rate of 2 megabits per second, just over 100 bytes will beavailable if, for example, eight lines of the vertical interval are usedfor these purposes. Given these 100 bytes per field, if this is carriedout over 15 consecutive fields, the requisite 1,500 bytes needed for thestorage of a single picture will be available, thereby enabling fourpictures to be recorded per second automatically, assuming compliancewith an interlaced NTSC standard of sixty fields per second. Thus,assuming sixteen sample pictures are to be displayed for a given lengthof program material, each sample picture automatically will be recordedin the blanking interval as just described, preferably with a period ofseveral minutes elapsing between the recording of each sample picture.

The vertical interval will be recorded more or less continuously withinformation associated with all of the pictures in the sequence. As itcomes time to record a sample picture, this new picture is added to theexisting sequence of information representative of all previouslysampled pictures. Thus, when information representative of sixteenpictures has been accumulated, four seconds of vertical retraceintervals will be required for the recording thereof, and, in thisparticular embodiment, this four seconds of vertical retrace informationsimply will be rewritten in a continuous cycle every four seconds, sincesixteen pictures were previously determined to be the total numberassociated with that program material. Thus, once these four secondshave been recorded on the tape, during playback and indexing, the systemneed only recover a total of four seconds worth of tape playback inorder to gather all of the information necessary to regenerate allsixteen of the pictures for display. Correspondingly, if indexing iscommenced at a point when, for example, only twelve of the pictures ofthe full capacity of pictures available for display on the screen of thedisplay device, then this system will only need to “back up” for threeseconds of tape playback in order to gain the information required togenerate and display the twelve sample pictures.

One of the principle advantages of the embodiment just described isthat, by using the same recording medium for both the program materialand the pictures representative of that program material, separatestorage media need not be required for the picture information. However,considering the fact that electronic hardware is required to derive andstore the picture information onto the recording medium, it may beadvantageous to add separate picture storage capabilities, particularlyin light of the operational features to be gained. One potentialdisadvantage of such separate storage, however, is that, should themedium carrying the program material be carried to a different machinefor playback purposes, unless the separate picture storage capability iscarried along therewith, the indexing capability according to theinvention will not be available during playback on this differentmachine. However, considering the fact that most consumers use the samesystem for recording and playback, the use of separate picture storageshould not represent a great disadvantage.

One potential problem, however, does exist in the case that a tape thatpreviously has been recorded is re-recorded, with new program materialinserted so as to replace one of the existing program. Here, the pictureindex information in parts of the tape will reflect the contents of theearlier recording, rather than that of the subsequent recording.Although this is likely to be an infrequent occurrence, it does suggesta preference for an embodiment which records the indexing information ata specific location, such as at the beginning of the tape, as this styleof embodiment will maintain the accuracy of the indexing informationeven when re-recording has occurred.

The following embodiments, then, include the use of a separate picturestorage capability, preferably in the form of a random-access memorysuch as dynamic RAM storage or “flash” memory, preferably used inconjunction with a battery back up facility of some kind to ensure thatthe indexing information is not lost during an interruption of power. Asan alternative, a non-volatile type of memory may be used, which may betransferred into a faster, volatile form of memory if retrieval is tooslow. The use of a separate memory for storing the picture informationmay be used with or without storage onto the medium storing the programmaterial itself. For example, information associated with each newpicture may be saved separately from the program recording, then, uponactivation of a particular operator command (such as END-RECORDING), theinformation may be transferred from the separate recording memory ontothe tape or disc in bulk fashion, in contrast to the accumulatingprocedure described earlier. For example, at the end of a recordingsession, one or more subsequent complete video frames (i.e., utilizingall of the available field lines, and not just the vertical blankinginterval) may be used to receive all of the picture information storedduring that recording session, then accessed when recording resumes orin conjunction with playback activities. Alternatively, upon thetermination of a recording session, the program storage mediumautomatically may be directed to a particular physical location whereinall of picture information is stored for later retrieval. In the case ofa video tape, for example, the tape may be rewound automatically, withthe picture indexing information always being stored at the beginning ofthe tape, so that it conveniently may be accessed when the tape is laterinserted into the VCR, thereby enabling the tape to be indexed prior tothe next viewing. While this presents certain disadvantages involvingthe length of time required to rewind the tape after its recordingsession, such a disadvantage will be minimized dramatically as theindustry moves from tape-based recording media to new video discsavailable from Sony and others, which promise to store many gigabytes ofinformation on an individual disc, and allow availability of thisinformation on a substantially random-access basis. With such media, itshould be a much more straightforward procedure to automaticallytranslate the appropriate recording head to a predetermined area of thedisc at the conclusion of a recording session, and then record thepicture indexing information.

As a further alternative to the use of separate picture storage inconjunction with the storage of program material on a high-capacitymedium, such separate memory may be made large enough to accommodatemultiple picture images associated with a number of different programstorage media, without requiring the transfer of the picture informationback onto the program storage media itself. That is, in thesealternative embodiments, each program storage medium will be identifiedin some manner and referenced to the separate picture memory, such that,after the recording has concluded, when accessed for playback theappropriate sections of the separate picture memory will be accessed,and the images associated therewith displayed. For example, in a VCRenvironment, the system may include several megabytes or gigabytes ofhard-disk RAM storage, which at 667 images per MB of storage, couldaccommodate hundreds or, perhaps thousands of index picturescorresponding to numerous program storage media. In such a case, duringthe recording process certain supplemental information may be recordedonto the tape along with the program material, such as a tape/disc orprogram storage medium identification number (recorded on a periodic orcontinuous basis), and/or frame indexing data, time codes or otherpertinent information facilitating more convenient access to particularportions of the program material. For example, in the case of a videocassette, while recording program source material, a unique cassetteidentification code and/or frame number simultaneously may be recordedonto the tape in a non-visible portion of the signal, such as thevertical blanking interval, or encoded into the control track, as inseveral indexing techniques well known in the art. At the same time,picture information associated with the program material, either derivedfrom the incoming source program and locally compressed or received inconjunction with the program material through a cooperative broadcasterproviding such information, simultaneously would be stored in theseparate picture storage medium along with the tape ID code and frameindexing information, if available. Upon playback or through theactivation of an INDEX command, under operator direction the storedpicture information would be accessed from the separate memory, andformatted for display purposes, preferably in the form of an array ofwindows, each containing a picture. However, by using a recordednumerical index, time code, or other means for advancing to a particularsection of the tape (or disc), an operator may then use a cursor,keyboard or any kind of pointing device to select one of the displayedpictures of the array, which would then cause the program storage mediumto advance to the section associated with that picture, to commencereplay therefrom, for example, or to begin recording over previouslystored program information no longer to be retained.

It should be mentioned, however, that other alternatives according tothis embodiment exist for the storage of the picture information orindex data to access such information. For example, information may beimprinted on, or encoded within the cassette shell itself, then read bythe apparatus of the invention as part of the display of the pictureinformation. In one arrangement according to this embodiment, cassetteshells may be provided incorporating semiconductor or other solid-statetype of memory, for example, bonded to the inside of the shell, withelectrical contacts provided on the outside of the shell for makingconnection to the IC chip. Upon insertion of the cassette into playbackequipment incorporating the invention, the picture information may bestored in the solid-state device and retained there until the next timeit is accessed or modified, during subsequent usage. Preferably, anypower required for such a memory chip would also be supplied by theplayback equipment receiving the cassette, and delivered to the devicethrough the contacts provided externally on the cassette shell. In lieuof a chip or other imprinted means storing the actual pictureinformation, a bar code, for example, may be used to identify thatparticular cassette and, optionally, to further encode areas associatedwith a separate memory within the equipment receiving the cassette orother medium, within which separate memory the actual pictureinformation is maintained.

FIG. 3 illustrates in block diagram form, a hardware embodiment of theinvention incorporating separate picture storage means. This figureillustrates a comprehensive version of the invention including featureswhich already may be present in other pieces of equipment, and thereforemay not need to be replicated. For example, although, in the preferredembodiment, the same device used to present the video program itself isused for the display of the pictures associated with the indexingthereof, this need not be the case in all embodiments. For example,given a different device, such as a computer, capable of receiving andreading the medium upon which the program material is stored, indexingmay be carried out in an entirely separate system, for example, throughthe use of the computer display. Also, a tuner 302 is shown, in theevent that source material is modulated in some form, requiring aconversion to baseband video. However, such a tuner already may bepresent in an existing VCR or other piece of viewing apparatus.Additionally, the program record block 310 may form part of an existingconventional VCR, in which case other components associated with theinvention may be housed in a separate enclosure and attached to the VCRas an add-on unit.

With the understanding that those modules already present in associatedequipment need not be duplicated, the system of FIG. 3 operates asfollows: an incoming video signal, either directly input along path 301,or converted by tuner 302, is placed onto a signal line 304 using aninput video switch 306 which, like most functional blocks in the system,is under the control of a central microprocessor 308. Thismicroprocessor 308 may be of conventional design, and preferably isimplemented in the form of a single-chip microcomputer having “on-board”temporary storage and read-only memory (ROM) used to store softwareprogram code. The video signal present on path 304 may be switcheddirectly to an output line 312 through output video switch 314, in theevent that a user simply wishes to view program source material withoutrecording or indexing of any kind. If the invention is implementedwithin a VCR, these video switches 306 and 314 already may be present.For that matter, the single chip microcomputer 308 may also beavailable, and if the unit has sufficient additional inputs, outputs,program memory and performance, also may be utilized for some or all ofthe functions of the invention which will now be described.

The incoming video signal on path 304 is presented to a mixer 320, apicture detector/generator 322, and, optionally, to a synchronizationpulse detector 324 which, in turn, feeds a signal to the microprocessor308. The sync detector 324 may be unnecessary, in the event that thepicture detector/generator 322 extracts sufficient information from theincoming video signal to perform all desired functions. The picturedetector/generator block 322 in the preferred embodiment is capableeither of detecting compressed picture information provided along withprogram material by a cooperating broadcaster, or alternatively samplesthe incoming video signal under microprocessor control and generates thepicture information directly from the video signal, in which case block322 is assumed to include appropriate digitizing and video datacompression means. Regardless of the source, the compressed pictureinformation is stored in the separate picture memory means 326 for laterretrieval during playback, this picture memory means 326 preferablybeing supported by back-up power source battery 328.

A video selector switch 330 is used to route information either from thepictured detector/generator block 322 or alternatively from the picturememory means 326, under microprocessor control in accordance with theselected operational mode. For example, if the picture index informationis generated locally and intended to be stored together with therecorded program information but without separate picture storage,switch 330 will be used to route the generated picture informationdirectly into the mixer 320, to be recorded simultaneously with theprogram material, in block 310. Alternatively, if the pictureinformation is recorded onto the program medium as part of a batchtransfer process, switch 330 may be used to form a path between picturememory means 326 and the mixer 320, so as to effect the transfer of agroup of pictures to the recording medium, again under microprocessorcontrol. To ensure that the picture information is properly recordedonto the program medium, the synchronization detector 324 may be usedfor such a purpose. In the event that the picture information is storedonly in picture memory means 326 and not as part of the program materialon the recording medium, the picture memory means 326 preferably alsowill record an identification code for the medium and, optionally, frameidentification information or time-code data used as indexing pointersto the picture stored in conjunction with a particular recordingsession. Since it will be advantageous to record at least such auxiliaryinformation into the record medium during storage of the programmaterial, switch 330 will route such information on a selective basisfrom a picture memory means 326 through mixer 320 for inclusion in therecorded video signal. Particularly, in the event that some form oftime-code is used for this purpose, a separate block 342 labeled “tapetravel control” may be used to keep track of the sequencing of the mainrecording medium (whether tape or disk), feeding a signal to the picturememory means 326 along signal path 343.

Regardless of where the picture indexing information has been stored,upon playback and/or entry of an “INDEX” command via remote control 350linked to microprocessor 308 through infrared detector 352, the systemcauses one or more of the stored pictures to be output through signalpath 312 in communication with a suitable display device (not shown). Asa first step toward displaying this information, the system willdetermine whether the picture information has been stored on the programmedium tape or disk, or instead has been stored separately in thepicture memory means 326. Typically this easily will be accommodated byinterrogating the program recording medium when it is inserted in theplayer unit or otherwise activated for access. If it is determined thata different section of the medium needs to be accessed in order torecover this information, the tape travel control block 342 may beutilized for such a purpose. If the system determines that, indeed,picture indexing information has been stored on the recording medium, itwill be extracted along path 360 and delivered to a videogenerator/frame buffer block 362, under control of the microprocessor308 to be properly formatted for display purposes. In the event that asingle array of windows is insufficient to display all of the imagesassociated with the recordings of particular program materials, or isincapable of representing the entire contents of the recording medium, anumber of multi-window displays may be used, for example, as “pages”,with the microprocessor facilitating such displays in conjunction withpicture memory means 326, video generating/frame buffer 362, and inputsfrom a user via remote control 350. As a further convenience, a printer390 may be attached through an I/O port 392, enabling an operator toprint out the pictures onto paper or a label for affixation to theprogram record medium or packaging.

If it is determined that the pictures associated with certain programmaterial have been recorded separately in picture memory means 326, thesystem only may need to interrogate the program recording medium todetermine an identification number and thereby locate the associatedpictures in memory means 326. Having located a correct set of indexingpictures, these pictures then may be displayed. As discussed above, inthe event that frame indexing, time codes, or the like have been used tocross reference recorded program material and the pictures associatedtherewith, once an array of pictures is provided to an associateddisplay device, touch-screen techniques may be used with appropriatedisplay facilities, or “point-and-shoot” pointing techniques may be usedin conjunction with the remote control 350, thereby enabling one of thedisplayed pictures to be pointed to and selected, following which thetape travel control block 342 will cause the recorded program to advanceor rewind to that portion of the tape associated with the pictureselected. Again, as utilized herein, “tape” travel control should beconstrued more broadly to include disks and any other form of recordingmedia.

As an alternative to the storage and display of still picturesassociated with the recorded program material, each window instead mayinclude motion imagery representing a few seconds of the recordedmaterial at that particular time. Since it is an object of thisinvention to enable an operator to more easily and conveniently locatepreviously recorded material, particularly when stored in an unlabeledor even mislabeled manner, use of motion pictures in each window mightfurther assist a viewer in determining the representative programmaterial.

In order to generate such motion imagery, many of the steps justdescribed would continue to be used, however, at the time associatedwith sampling the source material, not one but a few or numerous frameswould be recorded and later displayed in such a manner that each windowcould communicate a few seconds or more of “action” which would then berepeated, much as the movement of hurricanes and cloud formations isdepicted in televised satellite weather forecasts. So as to conservememory, consecutive frames would not be stored but, instead, a number offrames would be skipped between the updates. Although this might resultin uneven motion within a window, it would allow a much greater range ofdiscernability within a particular window for a given storagerequirements. In order to generate such windows with motion, picturesare retrieved from the memory means 326 and delivered to the videogenerator/frame buffer block 362, in accordance with a predeterminedupdate rate. In order to reduce bandwidth requirements along path 364,each displayed window may be updated on a sequential basis, resulting inthe movements in the various windows being “out-of-sync” with oneanother; however, it will be straightforward for the viewer to interpretthe images. That is, for example, the contents of picture memory means326 may be sampled with respect to the contents of 16 window No. 1,causing motion to appear in that window, while displaying all otherwindows as they were in the previous frames. Next, the memory means 326may be sampled and window No. 2 updated, causing relative motion thereinto be perceived, while continuing to present the information in allother windows without change, including window No. 1, and so on.

The ability to sample the memory means 326 in order to update windowshaving motion imagery enables the hardware of the system to performother advantageous functions not associated with video indexing. Forexample, the input video source may be sampled in a manner similar tothat just described in order to display multiple windows, each withmotion imagery, and each representative of a different incoming channel,yet requiring only a single tuner. In this a case, the system operatesas follows. Under microprocessor control, the tuner 302 is set to afirst channel, and information representative of the image on thatchannel may be loaded into the frame buffer aspect of functional block362, appearing in a first window on an associated display device due tothe action of the video display generator in block 362. At a short timelater, the microprocessor will tune tuner 302 to a second channel, andan image will be extracted therefrom to appear in a second window on thedisplay device, and so on, until multiple windows on the screensimultaneously display images from the various channels so tuned. Bycontinuing the process of channel sampling in a cyclical fashion, eachwindow may then represent not only a still picture of the program beingbroadcast on each channel, but may be updated in a manner similar to theuse of motion images for indexing purposes, as previously described,thereby resulting in multiple windows, each with moving picturesrepresentative of different channels, and all derived through the use ofa single tuner. The operator, through controller 350, interface 352, andcentral microprocessor 308, may select which of the various availablechannels to display, and may be selected in “touch-screen” or“point-and-click” fashion, as described herein above.

Another mode of operation made possible by the apparatus according tothe invention, is the ability to use images from the picture memory 326as opposed to the recorded program 310 to monitor the progress of aprogramming sequencing operation in excess of ordinary playback speed.For example, in the case of a video program being recorded onto a videotape, utilizing current technology, in order to monitor the progress ofa rewind or fast-forward operation, the playback head must remain incontact with the tape, with the result being a “chopped-up” screendisplay during such operations. Not only is this sheared displayannoying to a viewer, but it may also limit the speed with which thetape may be advanced or rewound, owing to the fact that the scan headmust be in contact with the tape to transfer the images to the display.

Utilizing the principles of the invention, however, rather than extractframes from the recorded program for display purposes duringfast-forward and rewind, so long as a correspondence between frames ismaintained between the program record block 310 and picture memory 326,frames representative of the stored video program may instead be useddirectly from the picture memory 326 during such operations, therebyobviating the need to use a playback head or other form of pick-upremaining in contact with the storage medium. In practice, usingfast-forward as an example, the playback head may be retracted from therecording medium, allowing it to travel at a high rate of transportspeed, while images are instead extracted from picture memory 326 andoutput through path 312 to appropriate display electronics, in the formof flashing images representative of the underlying program.Coordination between the recorded frames of the video program and thoseassociated with the program segments of the picture memory would bemanaged by tape travel control block 342 and control time-code detectblock 340 in conjunction with microprocessor 308.

As with other embodiments of the invention, the advantageous use of theapparatus of FIG. 3 just described may make use of a picture memory onthe same medium as that used to store the video program or may utilize aseparate medium, depending upon the desired implementation. Even in thecase of the video tape, the same medium may be used by storing theframes associated with the segments of the program outside of the mainrecording area, as in the control or audio tracks, enabling the tape tobe retracted from the helical-scan head during rewind/play-backoperations, while leaving a less restrictive sensor associated with theaudio or control tracks in place. Alternatively, the picture memory maybe implemented as an entirely different medium, such as a solid-state orother form of rotating magnetic or optical medium, in which case nothingother than transport elements would need to touch the media duringrewind/fast-forward operations.

1. An apparatus, comprising: a memory; a processor; instructions stored in the memory that, if executed by the processor, causes the apparatus to perform operations comprising: reading picture information including a subset of images of a plurality of images of program material, the image subset representing segments of program material separated in time, the subset of images being stored separately from the program material on a same random-access physical media; generating an output based on the subset of images, the output configured to display images of the separately stored subset of images in a plurality of respective windows on a display device; receiving a selection associated with an image of the subset of images; and outputting the program material to the display device beginning with the section of the program material associated with the selected image and continuing with subsequent sections.
 2. The apparatus of claim 1, further comprising generating the output as a plurality of pages to accommodate the plurality of respective windows.
 3. The apparatus of claim 1, further comprising directly accessing the program material in accordance with an association of the selection of one the subset of images and an index.
 4. The apparatus of claim 1, wherein the index is stored on the random-access physical media.
 5. The apparatus of claim 1, wherein audio information is stored in conjunction with an image of the subset; user-selecting a displayed image from the subset, and wherein the stored audio information is replayed corresponding to the selected image.
 6. The apparatus of claim 1, further comprising a remote control, wherein the selection is received from the remote control associated with the apparatus.
 7. The apparatus of claim 1, wherein at least one of window includes motion imagery.
 8. A video indexing method, comprising the steps of: storing a video program having time-sequential frames representative of motion imagery on a random-access computer readable medium; and separately storing a subset of the frames associated with segments of the video program that are spaced apart in time, wherein separately stored frames of the subset are suitable for display in a plurality of windows on a display device, wherein the subset of the frames associated with segments of the video program is stored separately from the video program and on the random-access computer readable media, and wherein at least one window is selectable by a user, and upon selection of a window, the video program is output for display on the display device.
 9. The method of claim 8, wherein the video program is output for display on the display device beginning with a segment of the video program associated with a frame displayed in the selected window and continuing with subsequent segments of the video program.
 10. The method of claim 8, wherein the subset of frames is derived by periodically sampling images of video program.
 11. The method of claim 8, further comprising: accessing the random-access computer readable medium to initiate a replay of the video program from a point in the video program corresponding to a selected window.
 12. The method of claim 8, further comprising: storing audio information in conjunction with an image of the subset; enabling a user-selectable image from the subset; and replaying the stored audio information corresponding to the user-selectable image.
 13. The method of claim 8, further comprising: analyzing the video program to determine the subset of the frames associated with the segments of the video program; and creating an index of the frames to the segments of the video program.
 14. The method of claim 13, wherein at least one window includes motion imagery.
 15. A tangible non-transitory computer readable medium having stored thereon a video program having time-sequential frames representative of motion imagery, and a separately stored thereon, a subset of frames associated with segments of the video program which are spaced apart in time, the computer readable medium having software code stored thereon for causing a processor to carry out a method, comprising: outputting separately stored frames of the subset for display in a plurality of separate windows on a display device, wherein the tangible non-transitory computer readable medium is a random-access media enabling direct access to the subset of frames associated with segments of the video program are randomly accessible by a computing device.
 16. The tangible non-transitory computer readable medium of claim 15, wherein a user-selection of a displayed image from the subset initiates a replay of the video program from a point in the video program corresponding to the selected image.
 17. The tangible non-transitory computer readable medium of claim 16, wherein the replay is initiated by directly accessing the video program corresponding to the selected image.
 18. The tangible non-transitory computer readable medium of claim 15, wherein audio information is stored in conjunction with an image of the subset, wherein a user-selection of a displayed image from the subset causes a replay of the stored audio information corresponding to the selected image.
 19. The tangible non-transitory computer readable medium of claim 15, wherein an index of the subset of frames to the segments of the video program is stored on the tangible non-transitory computer readable medium.
 20. The method of claim 15, wherein at least one window includes motion imagery. 